Wireless streaming environments present many challenges for the system designer. For instance, clients can have different display, power, communication, and computational capabilities. In addition, wireless communication links can have different maximum bandwidths, quality levels, and time-varying characteristics. A successful wireless video streaming system must be able to stream video to heterogeneous clients over time-varying wireless communication links, and this streaming must be performed in a scalable manner. Scalability is needed to enable streaming to a multitude of clients with different device capabilities.
In order to achieve scalability and efficiency in wireless streaming environments, one must be able to easily adapt or transcode the compressed video stream at intermediate network nodes. A transcoder takes a compressed video system as the input, then processes it to produce another compressed video stream as the output. Sample transcoding operations include bitrate reduction, rate shaping, spatial downsampling, frame rate reduction, and changing compression formats. Network transcoding can improve system scalability and efficiency, for example, by adapting the spatial resolution of a video stream for a particular client's display capabilities or by dynamically adjusting the bitrate of a video stream to match a wireless channel's time-varying characteristics.
While network transcoding facilitates scalability in video streaming systems, it also presents a number of challenges. First, while computationally efficient transcoding algorithms have been developed, even these are not well-suited for processing hundreds or thousands of streams at intermediate wired network nodes or even a few streams at intermediate low-power wireless networking relay nodes. Furthermore, many prior art network transcoding methods impose severe encryption, decryption, and re-encryption schemes for the streamed data.
More specifically, in conventional video streaming approaches employing application-level encryption, video is first encoded into a bitstream using interframe compression algorithms. These algorithms include, for example, the Moving Picture Experts Group (MPEG) standard, the International Telecommunications Union (ITU) standard, H.263, or intraframe compression algorithms such as, for example, the Joint Photographic Experts Group (JPEG) or JPEG2000 standards. The resulting bitstream is then encrypted, and the resulting encrypted stream is packetized and transmitted over the network using a transport protocol such as unreliable datagram protocol (UDP). Prior Art FIG. 1 is a block diagram 100 which illustrates the order in which conventional application-level encryption is performed (i.e. Encode 102, Encrypt 104 and Packetize 106). One difficulty with this conventional approach arises when a packet is lost. Specifically, error recovery is difficult because without the data from the lost packet, decryption and/or decoding may be difficult if not impossible.
Prior Art FIG. 2 is a block diagram 200 illustrating another conventional secure video streaming system that uses network-level encryption (i.e. Encode 202, Packetize 204, and Encrypt 206). The system of Prior Art FIG. 2 can use the same video compression algorithms as the system of Prior Art FIG. 1. However, in the system of Prior Art FIG. 2, the packetization can be performed in a manner that considers the content of the coded video and thus results in better error recovery, a concept known to the networking community as application-level framing. For example, a common approach is to use MPEG compression with the RTP transport protocol which is built on unreliable datagram protocol (UDP), RTP provides streaming parameters such as time stamps and suggests methods for packetizing MPEG payload data to ease error recovery in the case of lost or delayed packets. However, error recovery is still difficult and without data from a lost packet, decryption and/or decoding is still difficult if not impossible.
Both of the conventional approaches of Prior Art FIG. 1 and Prior Art FIG. 2 are secure in that they transport the video data in encrypted form. However, with these conventional approaches, if network transcoding is needed, it must be performed in accordance with the method of Prior Art FIG. 3. That is, as shown in block diagram 300, the necessary transcoding operation is a decrypt 302, decode 304, process 306, re-encode 308, and re-encrypt 310 process. As shown in the block diagram 400 of Prior Art FIG. 4, in another conventional approach, the computational requirements of the operation of Prior Art FIG. 3 are reduced to a decrypt 402, transcode 404, and re-encrypt 406 process. Specifically, this computational reduction is achieved by incorporating and efficient transcoding algorithm (i.e. transcode module 404) in place of the decode 304, process 306, and re-encode 308 modules of Prior Art FIG. 3. However, even such improved conventional transcoding algorithms have computational requirements that are not well-suited for transcoding many streams in a network node. Hence, conventional schemes employing such encryption methods further reduce computational and streaming efficiency.
As yet another concern, wireless streaming systems are limited by wireless bandwidth and client resources. Wireless bandwidth is scarce because of its shared nature and the fundamental limitations of wireless spectrum. Client resources are often practically limited by power constraints and by display, communication, and computational capabilities. As an example, wireless transmission and even wireless reception alone typically consume large power budgets. In order to make the most efficient use of wireless bandwidth and client resources, it is desirable to send clients the lowest bandwidth video streams that match their display and communication capabilities. In wireless streaming systems where a sender streams video to a number of heterogeneous clients with different resources, network transcoders can be used to help achieve end-to-end system efficiency and scalability.
In hybrid wired/wireless networks, it is often necessary to simultaneously stream video to fixed clients on a wired network and to mobile clients on a wireless network. In such a hybrid system, it may often be desirable to send a full-bandwidth, high-resolution video stream to the fixed wired client, and a lower-bandwidth, medium-resolution video stream to the mobile wireless receiver. Conventional video streaming approaches, however do not achieve the efficiency and scalability necessary to readily accommodate the video streaming corresponding to hybrid wired/wireless networks.
Yet another example of the drawbacks associated with conventional video streaming approaches is demonstrated in conjunction with wireless appliance networks. In many wireless appliance networks, mobile senders and receivers communicate with one another over wireless links. A sender's coverage area is limited by the power of the transmitted signal. Relay devices can be used to extend the wireless coverage area when intended receivers are beyond the immediate coverage area of the sender. However, in the case of heterogeneous clients within the same wireless network, it may be desired to provide a higher bandwidth, high-resolution video stream to the high power wireless receivers, and a lower bandwidth, low-resolution video stream to the low power wireless receivers. Once again, conventional video streaming approaches, however do not achieve the efficiency and scalability necessary to readily accommodate such video streaming demands in wireless appliance networks. Although the above-listed discussion specifically mentions the shortcomings of prior art approaches with respect to the streaming of video data, such shortcomings are not limited solely to the streaming of video data. Instead, the problems of the prior art span various types of media including, but not limited to, audio-based data, image-based data, graphic data, web page-based data, and the like.
Accordingly, what is needed is a method and/or system that can stream media data in a computationally efficient manner. What is also needed is a method and/or system that can satisfy the above need and that can also stream media data to heterogeneous clients (“receiving nodes”) that may have different display, power, communication and computational capabilities and characteristics. The present invention provides a novel solution to these needs.